Smartphones and other mobile devices are increasingly used for visual navigation. In visual navigation (VN) systems (VNS), visual-inertial-odometry (VIO), or Vision Based Positioning (VBP) systems (collectively referred to as “VNS” herein) images captured by a camera are used in conjunction with other sensor or wireless measurements for location determination and/or to estimate a trajectory of the mobile device.
The term “odometry” refers to the estimation of changes in position over time based on measurements from sensors. As used herein, the term “Visual Inertial Odometry” (VIO) refers to both odometry using a combination of images and other sensor based measurements as well odometry based on images. In VIO, several visual features may be tracked from frame to frame and used to determine an accurate estimate of relative camera motion.
In traditional VNS, visual or appearance based feature matching between images is used to determine a pose of the mobile device. Traditional VNS' suffer from “pose drift” (or drift), which describes the error between the actual position of the mobile device and a VN determined position of the mobile device. One approach to reducing or mitigating pose drift is “loop closure”. Pose drift or drift occurs because of residual errors in pose computation for an individual frame and from the accumulation of such errors from frame to frame. In loop closure, a mobile device may determine if, after some arbitrary length of time, a currently viewed feature corresponds to some previously viewed feature on its trajectory. The position and/or trajectory of the mobile device may then be corrected based on one or more of: current camera pose, the previously estimated location of the feature and/or the corresponding previous camera pose, thereby decreasing the impact of pose drift. However, as the length of time increases, the search space of feature points that need to be visually matched to determine if a loop closure event has occurred increases significantly—thereby limiting applicability. Moreover, because the appearance of a feature may change based on viewpoint and/or illumination, loop closure based solely on visual or appearance based feature matching may be challenging.